The present invention relates to an imaging apparatus such as electronic or digital still cameras and digital video cameras. Particularly, the present invention relates to an apparatus and a method for eliminating the noise of image data output by a photoelectric transducer element in an imaging apparatus of such an apparatus.
In digital still or video cameras, an optical image must be converted to an electronic image. A photoelectric transducer element such as a charge coupled device (CCD) is commonly used for such conversion.
In such a digital device, it is common for image data to be composed of separate color components such as red (R), green (G), blue (B), CMY, etc. A number of digital cameras use separate CCDs for each color, such as a CCD for red (R), a CCD for blue (B) and a CCD for green (G).
However, in order to reduce manufacturing costs and simplify equipment, many devices instead acquire image data for three primary colors of R, G and B using a single CCD. To acquire image data of the three primary colors using the one CCD, a method of attaching a mosaic optical filter to the single CCD is widely utilized. Such a filter is called a color filter array (CFA) and, when such a filter is attached to a CCD, data for each color is detected in a separate pixel. For example, data for red (R) may be detected in one pixel, and data for green (G) may be detected in the adjoining pixel.
The colors in a color filter may be arrayed in a variety of patterns with a Bayer color array being a common example of a typical array. FIG. 1 shows how colors are arranged in a Bayer array color filter.
As shown in FIG. 1, there are twice as many pixels of G as R or B in the Bayer array. This is because the G information is more important for human vision.
In an image processing unit for acquiring image data using a CCD, the mismatch of gain in an even line and in an odd line may occur.
This type of mismatch is especially significant in devices where two output terminals are provided for an even line and one output terminal is provided to an odd line of a CCD. The mismatch becomes remarkable because, when two output terminals are provided, a separate output is provided for respective output terminals and differences between amplifier gains may be reflected in their respective output signals. That is, a difference between amplifier gains of the amplifiers appear as a difference in magnitude between output signals.
Such mismatches may also occur in CCDs provided with only one output terminal. Generally, when the data of a pixel is output, the value of a pixel being output is often influenced by the value of the previously output pixel because of the amplification characteristics of the amplifier provided to the one output terminal and a mismatch may result.
In this case, a mismatch may occur because, if the data of a pixel is influenced by that of the previous pixel as described above, the value of a green (G) pixel in an odd line is influenced by the value of the adjacent red (R) pixel because of the characteristics of a Bayer array. Similarly, the value of a green (G) pixel in an even line is influenced by the value of the adjacent blue (B) pixel. Therefore, if, for example, a red object is photographed, green in an odd line will be strongly influenced by the adjacent red pixel, however, green in an even line will be little influenced by the adjacent pixel. As a result, the values for green pixels in the odd and even lines will differ and, as a result, a mismatch will occur.
A method for adjusting an amplifier and an attenuator for adjusting a difference between gains in odd and even lines in cameras with a CCD with two output terminals, as well as a method of storing a calibration value and correcting gains in an odd line and in an even line in image processing are known or proposed.
However, the factors leading to a CCD gain mismatch are often variable and inconstant; for example, the effect of the previous pixel in the same line varies greatly with temperature. It is therefore in practice very difficult to execute any of the above described methods and their application remains, for the most part, theoretical.
Noise caused by the mismatch of gains in odd and even lines commonly causes lateral stripes in a final image. Further, interpolation, edge highlighting processing, or other processes applied to the image containing these lateral stripes may compound the negative effects. The quality of a final image may be severely deteriorated.
As such differences between gains in odd and even lines may be regarded as striped noise, the elimination of the striped noise by noise elimination processing in the final step of image processing may be also taken into consideration. However, there is then a problem that a considerable amount of time is required for image processing and a further problem that the edge of an image may blur because of the side effects of noise elimination.
The present invention is made to solve the problems outlined above and has an object of providing a noise elimination method and apparatus which can eliminate noise caused by the mismatch of odd and even line gains from image data acquired from a CCD provided with a color filter. The present invention is characterized in that, in order to solve the above problems, components as described below are included in a noise elimination apparatus in order to eliminate the noise of Bayer-type image data output by a photoelectric transducer element provided with Bayer-type color filter.
That is, the present invention is characterized in that the noise elimination apparatus includes high frequency component quantity detecting means for detecting the value of a high frequency component in each pixel of the Bayer-type image data, graduation quantity calculating means for calculating the quantity of graduation in each pixel of the above Bayer-type image data, and optimum graduation quantity determining means for changing the above quantity of graduation to an optimum value based upon the value of a high frequency component detected by the high frequency component quantity detecting means and then adding the changed quantity of graduation to each pixel.
By adding an optimum quantity of graduation based upon the value of a high frequency component of each pixel, the deterioration of image quality can be reduced.
The present invention may also be characterized in that the above high frequency component quantity detecting means may include a Laplacian filter to which the value of a target pixel and the values of the four pixels immediately adjacent to the target pixel are input. A signal output from the above Laplacian filter is output as the value of a high frequency component. The input of these pixel values enables the reduction of the effect of noise on each line.
The high frequency component quantity detecting means of the present invention may also comprise a Laplacian filter to which the value of a target pixel and of the four immediately adjacent pixels are input, upper and lower difference calculating means for outputting the absolute value of difference between the values of the upper and lower adjacent pixels to the target pixel, right and left difference calculating means for outputting the absolute value of difference between the values of the pixels to the right and left of the target pixel, and addition means for weighting and adding a signal output from the above Laplacian filter, a signal output from the above upper and lower difference calculating means and a signal output from the above right and left difference calculating means. A signal output from the above addition means is output as the value of a high frequency component. This configuration reduces the effect of noise on each line and thereby reduces, or eliminates, the resulting deterioration of image quality.
The graduation quantity calculating means of the present invention may include mean value calculating means for acquiring the weighted mean value of the value of a target pixel and the values of pixels around the target pixel and differential value calculating means for outputting a differential value between the above weighted mean value and the value of the target pixel. A signal output from the above differential value calculating means is output as the quantity of graduation.
As difference is output as the quantity of graduation, the quantity of graduation according to the quality of an image can be calculated.
Another aspect of the present invention relates to a noise elimination apparatus characterized in that optimum graduation quantity determining means as described above further includes conversion means for converting the value of a high frequency component to a scale factor, multiplication means for multiplying the scale factor and the quantity of graduation, and addition means for adding the quantity of graduation multiplied by the scale factor to the value of a target pixel. As a scale factor varies according to the value of a high frequency component, noise can be eliminated while image quality is maintained.
A still further aspect of the present invention is in the form of a noise elimination method for eliminating the noise in Bayer-type image data output by a photoelectric transducer element provided with Bayer-type color filter. In such a method, the value of a high frequency component in each pixel of Bayer-type image data is found, a graduation quantity calculation step calculates the quantity of the graduation of each pixel constituting Bayer-type image data, and an optimum graduation quantity determination step changes the quantity of graduation to an optimum quantity based upon the high frequency component value detected in the high frequency component quantity detection step and adding the changed quantity of graduation to the value of each pixel are included.
As the quantity of graduation added is based upon the value of a high frequency component of a pixel, the deterioration of the quality of an image can be reduced.
Yet another aspect of the present invention is characterized in that the high frequency component quantity detection step includes a Laplacian filtering step for calculating the quadratic differential value of the value of the following target pixel based upon the value of a target pixel and the values of each of the four immediately adjacent pixels. The quadratic differential value is calculated as the value of a high frequency component.
As the values of all adjacent pixels are input, the effect of noise on each line can be reduced.
The present invention may also be configured so as to comprise a high frequency component quantity detection step including a Laplacian filtering step for inputting the value of a target pixel and the values of the four immediately adjacent pixels and calculating the quadratic differential value of the value of the target pixel, an upper and lower difference calculation step for calculating the absolute value of difference between the values of the pixels immediately above and below the target pixel, a right and left difference calculation step for calculating the absolute value of difference between the values of the pixels to the immediate right and left of the target pixel, and an addition step for weighting and adding the quadratic differential value, a signal output in the upper and lower difference calculation step, and a signal output in the right and left difference calculation step. A signal output in the addition step is output as the value of a high frequency component.
With such a configuration the deterioration of image quality is reduced because the difference between pixel values is referred to in order to reduce the effect of noise on each line.
The present invention may also be comprised so that the graduation quantity calculation step includes a mean value calculation step for acquiring the weighted mean value of the value of a target pixel and the values of surrounding pixels and a differential value calculation step for calculating a differential value between the weighted mean value and the value of the target pixel. A signal output from the differential value calculation step is used to calculate the quantity of graduation. As difference is output as the quantity of graduation, the quantity of graduation according to image quality can be calculated.
The present invention further relates to a noise elimination method characterized in that the optimum graduation quantity determination step includes a conversion step for converting the value of a high frequency component to a scale factor, a multiplication step for multiplying the scale factor and the quantity of graduation, and an addition step for adding the quantity of graduation multiplied by the scale factor to the value of a target pixel.
As a scale factor varies according to the value of a high frequency component, noise can be eliminated while maintaining image quality.